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Abstract 

For extended autonomous operation , rovers must iden- 
tify potential faults to determine whether its execution 
needs to be halted or not At the same time f rovers 
present particular challenges for state estimation tech- 
niques: they are subject to environmental influences 
that affect sensor readings during normal and anoma- 
lous operation , and the sensors fluctuate rapidly both 
because of noise and because of the dynamics of the 
rover ’s interaction with its environment. This paper 
presents MaKSI, an on-board method for state esti- 
mation and fault diagnosis that is particularly appro- 
priate for rovers . The method is based on a combi- 
nation of continuous state estimation , using Kalman 
filterSf and discrete state estimation , using a Markov- 
model representation. 

1 Introduction 

Rovers that operate autonomously for extended pe- 
riods of time must be able to detect and diagnose 
anomalous situations and recover from faults that 
do not require ground-operator intervention. Rovers 
present characteristics that make this problem chal- 
lenging. They receive streams of continuous-valued 
sensor data that fluctuate with noise and environmen- 
tal interactions. FYom this data they must infer the 
presence of nominal and off-nominal states, but these 
states depend on the situation. The boundaries of 
states can change depending on the context: for ex- 
ample, a high current driving uphill or over an obsta- 
cle may be normal, but a high current on smooth, flat 
ground may indicate an anomaly. In addition, the set 
of anomalous states can change: for example, the set 
of possible anomalies will be different for driving and 
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taking pictures. Moreover, rovers are limited in power 
and weight, which in turn limits processor speed and 
memory. They must therefore use computationally ef- 
ficient procedures for state estimation. 

The rover state can be thought of as transitioning 
among a set of possible, qualitatively different states. 
These states may correspond to operational modes of 
the rover (e.g., driving) or fault modes (e.g., broken 
wheel gear) 1 . Transitions may be explicit, based on 
actions taken by the rover executive (e.g., stopped to 
driving), or implicit, based on sensor information (e.g., 
wheel encoder nominal to broken). 

Sensor failures may be inferred from the diagno- 
sis, and a failure should then influence future state 
estimation and diagnosis. For example, driving with 
a broken encoder will give rise to different “normal” 
sensor readings for the broken encoder. 

Two major branches of work for state estimation 
and fault diagnosis are Kalman filters from control 
theory and qualitative model-based diagnosis from ar- 
tificial intelligence. State estimation for rovers exceeds 
the capabilities of the current approaches. 

Qualitative model-based techniques for diagnosis 
[2, 6] rely on the system transitioning occasionally 
from one steady state to another. Rovers receive 
rapidly-changing streams of continuous- valued sensor 
data. In addition, the model-based techniques often 
rely on a snapshot of the system, disregarding his- 
tory. But in fact the history may be critical to reach a 
correct diagnosis; the probability of a particular fail- 
ure may be significantly different based on the prior 
rover state. In addition, the qualitative approaches 
rely on global consistency, which can be expensive to 
compute, to compensate for the local inaccuracy of a 
qualitative model. 

Techniques for state estimation of continuous val- 
ues, such as Kalman filters [3], can track multiple hy- 

l In fact, the complete set of states is the cross product of 
the operational modes with the powerset of the possible fault 
modes. For modeling reasons some states are often combined 
with equivalent or similar states. 


potheses [7, 10], but they lack methods for automa- 
tially choosing which states to track. Tracking all pos- 
sible states is infeasible. 

In this paper, we present MaKSI (Markov And 
Kalman State Identification), which combines continu- 
ous probabilistic state estimation using Kalman filters 
(KFs) with discrete qualitative state estimation us- 
ing a Markov-model representation [4]. The discrete 
states correspond to qualitatively different modes of 
the rover (driving nominally, idle, stuck wheel, etc), 
to each of which is associated a model of operation 
represented as KF parameters. The development of 
MaKSI arose from our experience with model-based 
techniques in the NASA Ames 1999 Marskohod field 
test, and it borrows some ideas from the model-based 
techniques. Differences include the addition of quanti- 
tative information through KFs, context-specific prob- 
abilities of state transitions, and guarantees on com- 
putational efficiency. 

On-boaxd techniques must work within constraints 
on computation and memory. MaKSI builds on our 
own and others’ work on efficiently tracking belief 
states [9, 1] and continuous variables [7]. 

2 Combining discrete and con- 
tinuous state estimation 

The MaKSI approach to state estimation is built from 
Kalman filtering and Markov-model representations. 
Before discussing the complete approach, we briefly 
introduce the components from which it is built. 

2.1 Kalman filtering 

The Kalman filter (KF) is designed to estimate the 
state of a process given observations. Here we con- 
sider the standard, discrete KF, which is sufficient for 
our initial experiments. More complex KFs can be for- 
mulated for nonlinear processes and observations and 
for continuous time. In the standard, discrete KF, the 
process is assumed to evolve linearly given the previ- 
ous state and a control input: 

x t + 1 = A t x t -1- Bu t + w t (1) 

where x t is the state (vector) at time t, u t is the con- 
trol input, and w t is white, normally-distributed noise. 
The process is observed through measurements related 
linearly to the process state: 

z t = H t x t 4- v t (2) 

where z t is the measurement (vector) and v t is white, 
normally-distributed noise. 


At each time step, the estimate of the state is first 
updated using information about the state model, the 
control input, and the process noise. This estimate 
(the a priori state estimate, xj) is then fed through 
the observation model. The observation model is used 
to predict what observations should be seen. These 
are compared against the actual observations, and the 
difference is used to modify the a priori state estimate, 
arriving at the new, a posteriori state estimate, x+ . 

The observation noise, in the form of a covariance 
matrix, is used to update the Kalman gain matrix , 
which is the weighting factor used to combine the a 
priori state estimate and the observation differences. 
The gain matrix also depends on the state error co- 
variance matrix, which is defined as: 

P t = £[(x t -i e )(*t-i t ) T ] (3) 

The error covariance matrices are updated over time. 

The KF finds the optimal estimate of the process 
state (under the assumptions of the model), in that 
it minimizes the expected least-squares error. Briefly, 
the filter operates by first predicting the state using 
the state update equations, then correcting the pre- 
diction using an observation. Error covariances of the 
state and the observation contribute to “weight” the 
model, balancing the state model (the prediction) and 
the observation. 

Details of KFs can be found in [5, 3], among others. 

2.2 Markov-model representation 

While Kalman filters handle the problem of state es- 
timation in a continuous space, they do not offer any 
assistance when the state branches into two or more 
qualitatively different states. In contrast, the stan- 
dard model for Markov decision processes (MDPs) is 
a set of discrete states with probabilistic transitions 
among the states. In the case of partially-observable 
MDPs (POMDPs), the estimation of the current sys- 
tem state is represented by a probability distribution 
over the set of discrete states. 

A standard POMPD model consists of a set of 
states, 5, a set of actions A, and a set of observa- 
tions 0, The model also contains a transition matrix 
T, of size |S| x |S| x |A|, where Ujk is the probability 
of transitioning from state s* to state 3j when action 
a* is chosen. In a standard POMDP model, there 
is an observation matrix Q of size |5| x |0|, where 
qij is the probability of seeing observation Oj when in 
state In MaKSI, we do not have this direct in- 
formation, but we use an indirect calculation via the 
Kalman state estimate. 



Finally, the probability distribution over states at 
time t is denoted as i r(£), where 7Tj(£) is the probabil- 
ity that the true state is Si given information about 
actions and observations. 

Given an observation o } and an action a*, the state 
distribution is updated according to the following for- 
mula: 
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As mentioned earlier, we modify this (see the following 
subsection) to use an indirect computation of qij. 

Since the goal of state identification is to find the 
best estimation of the current state as a passive op- 
eration, the parts of the POMDP model for control 
(rewards and policies) are not applicable to this prob- 
lem. 


2.3 Combining discrete and continu- 
ous models 


The basic idea behind MaKSI is to consider the sys- 
tem as a set of discrete states, but rather than treat- 
ing each as a static situation, the dynamics within 
the state sure represented using a Kalman filter. We 
start from the POMDP model and augment it with el- 
ements from the KF model. We use the term discrete 
state to represent the qualitatively distinct, POMDP- 
level discrete state of the system, and the term system 
state to represent the state vector of the actual system 
parameters at the KF level. 

As in the POMDP model, MaKSI represents the 
system as a set of discrete states 5 and a set of discrete 
actions A . We define a set of transition actions © as: 
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space V(s) of possible system state values. In gen- 
eral the value space can be any subset of the possible 
system state values; for efficiency, we restrict the con- 
straints to be univariate and linear, defining a (poten- 
tially infinite) hypercube value space. The KF model 
has a state and observation vector of the same length, 
where state element i[i] is the best estimate of the 
value observed as observation element o[i ] . 

Unlike the standard POMDP model, the observa- 
tion probability matrix Q cannot be statically pre- 
dicted from the discrete state. Instead, we approxi- 
mate this by an indirect computation via the KF state: 

« Prob(Oj\K) • Prob{K\si) (5) 

The element Prob(oj\K) is itself approximated as the 
volume of the multidimensional normal distribution 
function, described by the Kalman estimated state 
and state error covariance, beyond Oj (more precisely, 
1— the volume of the minimum error ellipse enclosing 
Oj). In the multivariate case this is further approxi- 
mated as a product over individual dimensions: 

Prob(oj\K ) » H (6) 

l<i<n 

where n is the length of any observation vector, Oj[i] 
is the ith element of the observation vector Oj, x(K) 
is the KF estimated state, P(K) is the KF state er- 
ror covariance matrix, and /() is the probability that 
an individual observation element is predicted by the 
state estimate and standard deviation in that dimen- 
sion 3 . The element Prob(K\si) is the volume of the 
Kalman-described normal distribution function that 
falls within the value space of s^; this is approximated 
for computational efficiency as a product of the indi- 
vidual dimensions: 

Pnb(K\si) * n (7) 

l<»< n 


These correspond to starting action a, ending action 
a, and the null action (am implicit, data-driven transi- 
tion). Similar to the POMDP model, we have a tran- 
sition matrix T of size |5| x \S\ x |0|, where Ujk is 
the probability of transitioning from discrete state 
to state $j when tramsition action 0* is taken 3 . Since 
there is a difference between tramsitioning from s» to Si 
and remaining in state (see below), we consider the 
probability of remaining in state Si given transition 
action 9 k to be 1 - ^ l<i<15 , 

Each discrete state s has associated with it a KF 
model and a set of constraints describing the value 

3 Note that the implicit transition* are thus dependent on the 
time step. See Section 4 for possible remedies for this. 


The normal distribution computations are performed 
efficiently by table lookup and interpolation. 

Using the approximation to the observation prob- 
ability q ^ , we then plug that back into the POMDP 
state distribution update function (Eq. 4) and use 
that to arrive at the new probability distribution over 
possible states. 

The update function can be seen intuitively as 
combining context-specific probability (the transition 
matrix), data-model compatibility ( Prob(K\si)) : and 
model predictiveness ( Prob(oj\K )). So a discrete state 
with a high probability in the state distribution will 

3 This assumes independence of the observation vectors, 
which is a simplifying, but not necessarily accurate assumption. 


be appropriate for the context, its model will produce 
a state estimate that is highly compatible with the 
value space constraints, and the observations will be 
highly consistent with the state estimate. 

One complication of using dynamic state models 
is that the KF system state depends on the initial 
conditions of the model. As the system predicts a 
transition from one discrete state to another, the sys- 
tem state in the new discrete state must inherit the 
system state from the previous state. Thus not all 
instances of a discrete state are equivalent. For ex- 
ample, consider two discrete states, one with wheel 
current rising, a second with steady wheel current. A 
later transition from the rising to steady will imply 
a higher steady wheel current than an earlier transi- 
tion. This is an important difference from standard 
POMDP-model representations, introducing the dan- 
ger of model explosion. 

MaKSI limits the explosion by maintaining a 
constant-size distribution over discrete states. This 
makes the POMDP state distribution update constant 
with respect to the number of states [9] 4 . The KF up- 
dates involve matrix multiplications and inverses, so 
they are of order 0(n 3 ) y where n is the size of the KF 
system state vector 5 . If the system state vector can be 
decomposed into independent subsets, the KF update 
depends on the size of the largest subset. 

The danger of limiting the discrete state distribu- 
tion to a constant size is that the state distribution 
update formula may produce a null distribution, with 
all elements 0. We have shown in [8] methods for over- 
coming this problem; these are not in our current pro- 
totype implementation, but they will be incorporated 
into the final implementation. 

We thus have a way of trading off computational 
complexity for model accuracy. The size of the dis- 
crete state distribution can be tailored to the compu- 
tational constraints of the application; in planetary 
rovers, with their relatively impoverished computa- 
tional power, this may be essential. 

3 Experimental validation 

We have constructed a prototype implementation of 
MaKSI and tested it on telemetry data gathered from 

4 This depends both on a constant bound on the set of tran- 
sitions from any state and on the fact that each instance of a 
discrete state is in general distinct from other instances, so that 
9tate transitions are treated as a chain without loops (except 
self-loops). 

5 The theoretical complexity for matrix multiplication and 
inverse is 0(n 3 80T ), but for the small matrix sizes we use, 
straightforward methods are more appropriate. 


the Marsokhod rover. The Marsokhod is a medium- 
sized planetary rover built on a Russian chassis; it 
has been demonstrated at field tests from 1993-99 in 
Russia, Hawaii, and deserts of Arizona and Califor- 
nia. The rover has six wheels, independently driven. 
For the experiments, the right rear wheel had a bro- 
ken gear, so it rolled passively. The Marsokhod is 
instrumented with sensors that measure body, arm, 
and pan/tilt geometry, wheel odometry and currents, 
and battery currents. 

The data used in this paper were collected in an 
outdoor “sandbox,” which is a gravel and sand area 
about 20m x 20m, with assorted rocks and small hills. 
This space is used to perform small-scale tests in a rea- 
sonable approximation of a planetary (Martian) envi- 
ronment. The only actions for this preliminary exper- 
iment were driving commands, in arbitrary directions 
and with varying topography. 

For the prototype system, we used wheel current 
and wheel speed (differences between successive en- 
coder values), along with the variation for current and 
speed (absolute value of differences between successive 
values). We ran 6 independent state identification pro- 
cesses, one per wheel. Each state identification process 
had identical starting conditions, so the only difference 
was in the data each received. 

The discrete states corresponded to idle states, 
driving states, and intermediate states (ramping up 
current and speed at the beginning of an action, drop- 
ping current and speed at the end of an action). These 
states were replicated for a small set of fault modes, 
including a stalled motor, a broken gear, and a broken 
gear and a broken encoder (the latter of these was in 
fact the case on the right rear wheel). The KF models 
were constructed crudely and by hand to model the 
approximate dynamics in each situation; more care- 
ful and detailed models should lead to more accurate 
state identification, but the hypothesis was that this 
would be sufficient for the prototype test. 

See Figure 1 for data from one such experiment, 
which we will call Experiment 1. States 0-2 are non- 
commanded idle and transition states, states 3-8 are 
normal driving states (state 6 in particular is the 
steady-state driving condition), and states 9-22 are 
error states (19-22 are the broken gear and broken 
encoder condition). In general the highest-probability 
state corresponds to the correct state. The broken 
wheel is correctly diagnosed (because of the lack of 
variation in the current); this requires a number of 
data points to accumulate enough evidence to sway 
the model (since the a priori probability of this event 
is relatively small). There is one error: in the left rear 



Figure 1: State estimates and wheel currents for each 
wheel, Experiment 1. The highest probability state 
is in fact the correct one except for a momentary 
misidentification in the left rear wheel. 


wheel, an interval of non-varying motor current leads 
to a momentary shift of belief towards a fault state, in 
this case a broken wheel. When the data vary again, 
the belief in the broken wheel disappears as well. 

Although the preliminary results have been encour- 
aging, a handful of cases cause problems. Figure 2 
shows one test case ( Experiment 2) where, for an un- 
known reason, the wheel currents flattened out for an 
extended period. Of the 5 working wheels, 4 incor- 
rectly identified a fault state during this interval. The 
broken wheel correctly identified the fault, except for 
one interval where its current showed some variation 
(again, for an unknown reason). This could be avoided 
by tweaking the probabilities of the transition to the 
fault condition, but the purpose of the experiment was 
to test intuitively reasonable models rather than time 
the models for a set of test data. 


Figure 2: State estimates and wheel currents for each 
wheel, Experiment 2. Anomalies in the data give rise 
to state identification errors. A broken gear is found 
as the most likely state when the wheel currents flat- 
ten out. The broken wheel is momentarily labeled as 
working when the data vary unexpectedly. 

Of the 50 test cases that we have gathered to date, 
a few also produced null distributions. This can arise 
from modeling errors; for example, we didn’t have a 
model for a dead motor controller, which occurred a 
number of times. It can also be a result of the belief 
state truncation, which demonstrates the need to add 
mechanisms to avoid that, as discussed in Section 2.3. 

The computational performance of the state iden- 
tification program is promising. The prototype state 
identification system is coded in Java for ease of visu- 
alization and rapid development. The algorithms are 
straightforward and not refined for efficiency purposes; 
we would expect significantly higher performance from 
a careful C/C++ implementation. On the other hand, 
the processor used for the prototype tests is a Sun 
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Figure 3: Elapsed time to update belief state for all 6 
wheels, Experiment 1. 

Ultra-2 (266 MHz), which is greater than what we 
can expect for near-term flight processors, but it is of 
the same order as the prototype_rover platforms we 
use for development and testing. 

The time required for the state estimation process 
in experiment 1 can be seen in Figure 3, This is the 
elapsed time of the updates for all 6 wheels, for each 
of which there is a belief state of size 16 (hence 16 
KFs per wheel, for a total of 96 KFs updated at each 
step). The update time is consistently just under a 
second while the rover is active, and less when it is 
idle. 

4 Discussion 

We have demonstrated an approach for computation- 
ally efficient state and fault identification that is par- 
ticularly appropriate for the dynamic environment and 
noisy data encountered by an outdoor mobile robot. 
We envision this approach as a prototype for future 
planetary rovers, where accurate and efficient state 
identification is a critical element of long-term au- 
tonomous operation. 

The approach presented here is a preliminary at- 
tempt to model and represent the states encountered 
by a rover. An obvious refinement of the work is to de- 
velop more careful and complete models of rover oper- 
ations. Although the crude models produce generally 
reliable state identification, they are imperfect; more 
precise models should lead to more accurate identifi- 
cation. Also, the KF model used for the prototype is a 
standard, discrete-time KF. This is a reasonable first 
approximation, but a more sophisticated KF model 
would support the more accurate models needed of 
the rover. 

A KF model has a large number of parameters that 
can be adjusted. In the prototype, these were set to 
intuitively reasonable values by hand. These could 
be inferred from a larger corpus of experimental data. 
Additionally, the transition probabilities of the dis- 


crete states were equally hand-set. Anomalies such as 
the broken-wheel misidentification could be reduced 
by tweaking parameters, but ideally this would be in- 
formation gathered from long-term experience with a 
platform (to gather reasonable fault probabilities). 

As mentioned in Section 2.3, the approach needs 
to be extended to handle the case where a null dis- 
tribution may arise. This should be a straightforward 
application of the work in [8]. 

Finally, the underlying probabilistic reasoning re- 
lies on a number of approximations. Understanding 
the relationship of the approximations to the com- 
plete, accurate probabilities is an important piece of 
understanding the quality of the approach as a whole. 
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